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May  30,  1980 

The  Honorable  William  V.  Roth,  Jr. 

Chairman,  Committee  on  Governmental  Affairs 
United  States  Senate 

Dear  Mr.  Chairman: 

In  your  letter  of  November  6, 1984,  you  requested  information  on  (1) 
the  Sergeant  York’s  unique  accelerated  acquisition  strategy  and  its  suit¬ 
ability  for  use  in  future  programs,  (2)  whether  the  Sergeant  York’s  orig¬ 
inal  mission  and  operational  requirements  had  been  changed  in  light  of 
the  weapon’s  performance  in  testing,  (3)  the  protection  against  cost 
growth  offered  by  the  warranty  provisions  of  the  Army’s  contract  with 
Ford  Aerospace  and  Communications  Corporation,  particularly 
regarding  technical  performance  problems,  (4)  the  weapon’s  achieve¬ 
ments  in  certain  specific  areas  during  testing,  and  (5)  the  factors  consid¬ 
ered  in  the  source  selection  process  that  had  led  to  awarding  Ford 
Aerospace  the  contract  to  complete  development  and  produce  the 
weapon  system. 

On  August  27, 1985,  the  Secretary  of  Defense  announced  his  decision  to 
cancel  the  Sergeant  York  program  after  only  64  of  the  planned  procure¬ 
ment  of  614  weapon  systems  had  been  delivered.  The  latest  operational 
tests  had  shown  the  Sergeant  York  to  represent  only  a  marginal 
improvement  over  other  available  air  defense  systems  in  countering  the 
threat.  Therefore,  the  Secretary  believed  that  completing  the  program 
was  not  worth  the  investment. 

In  view  of  the  Secretary’s  action,  we  discussed  your  request  with  your 
office  to  determine  which  of  your  concerns  would  be  of  continuing 
interest  to  you.  Based  on  these  discussions,  we  agreed  that  we  should 
focus  on  the  first  concern,  that  of  the  acquisition  strategy,  but  include 
such  information  on  the  warranty  provisions  and  the  Sergeant  York’s 
testing  history  as  was  necessary  to  assess  the  strategy’s  effect  on  the 
Sergeant  York  program. 

The  Sergeant  York  program’s  acquisition  strategy  contained  elements 
which  succeeded  in  controlling  costs,  but  it  would  not  have  achieved  the 
desired  early  deployment  even  if  the  program  had  continued.  Army 
managers  had  hoped  to  accelerate  the  Sergeant  York’s  deployment  by 
scheduling  less  testing  prior  to  production  than  is  normally  conducted 
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under  conventional  acquisition  strategies.  As  a  result,  critical  informa¬ 
tion  about  shortcomings  in  the  Sergeant  York’s  ability  to  perform  under 
realistic  battlefield  conditions  was  unavailable  to  decisionmakers  until 
after  the  system  had  entered  its  fourth  year  of  production  In  addition, 
surveillance  over  the  weapon’s  early  development  was  limited  to  a  few 
officials.  This,  together  with  the  Army’s  determination  to  come  as  close 
as  possible  to  meeting  the  schedule,  in  our  opinion,  created  an  atmo¬ 
sphere  of  unwarranted  optimism  about  the  prospects  of  overcoming 
some  serious  technical  shortcomings  disclosed  m  the  weapon’s  tests.  In 
responses  to  our  prior  reports  in  which  we  urged  caution  in  proceeding 
with  Sergeant  York’s  production,  the  Department  of  Defense  (dod)  main¬ 
tained  that  technical  risks  were  not  so  great  as  to  warrant  delays  and 
that  cost  savings  would  result  by  maintaining  the  schedule. 

We  believe  the  Army  should  continue  to  try  to  reduce  the  time  it  takes  to 
prepare  weapon  systems  for  deployment.  However,  we  believe  that  if 
the  Army  is  going  to  apply  the  type  of  strategy  used  in  the  Sergeant 
York  program  to  future  programs,  certain  conditions  should  be  present 
and  certain  actions  taken  to  provide  greater  assurance  that  the  strategy 
will  succeed.  We  are  making  some  recommendations  along  these  lines  to 
the  Secretary  of  Defense. 


Background 


I 


The  Sergeant  York  air  defense  gun  system  was  being  developed  to  fill  a 
void  in  the  Army’s  air  defense  in  the  forward  battle  area.  The  system 
was  to  engage  enemy  helicopters  and  fixed-wing  aircraft  used  m  close 
air  support.  It  was  to  have  a  three-member  crew,  and  its  armament 
included  twin  40-mm.  radar-directed  computer-controlled  guns.  In  all, 
614  gun  systems  were  to  be  procured  at  a  cost  of  $4.04  billion. 

The  Sergeant  York  was  to  be  acquired  under  an  accelerated  strategy,  the 
objective  being  to  field  the  weapon  as  quickly  as  possible  with  substan¬ 
tial  cost  savings.  This  accelerated  approach  featured  parallel  develop¬ 
ment  by  two  competing  contractors;  a  “skunk  works’’  approach  to 
contracting,  which  gave  each  contractor  the  flexibility  to  trade  off  some 
requirements  in  order  to  lower  costs;  a  short  combined  development  and 
operational  test;  and  a  concurrent  follow-on  development  and  initial 
production  phase.  The  strategy  also  emphasized  minimum  government 
management  during  system  development  and  restricted  access  to  con¬ 
tractors  and  contractor  information  to  avoid  leaking  proprietary  or  com¬ 
petition-sensitive  information  and  to  reduce  the  likelihood  of  a  protest 
from  the  contractor  not  selected  for  production. 
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The  Army  recognized  that  the  Sergeant  York’s  accelerated  strategy  car¬ 
ried  higher  than  usual  risks.  It  sought  to  minimize  them  and  thereby 
guard  against  cost  growth  by  providing  safeguards,  such  as  requiring 
the  contractors  to  use  mature,  proven  subsystems  in  the  weapon’s 
design  and  by  including  more  extensive  warranty  provisions  in  the  Ser¬ 
geant  York  contract  than  were  common  in  other  major  weapons  con¬ 
tracts.  In  addition,  the  negotiation  of  a  fixed-price  development 
contract,  which  included  three  fixed-price  production  options,  was 
intended  to  provide  further  protection  against  cost  growth  during  pro¬ 
duction  when  only  one  contractor  would  be  producing  the  weapon. 

The  Sergeant  York’s  acquisition  strategy  was  to  have  three  phases:  (1) 
engineering  development,  which  included  a  29-month  competition 
between  two  contractors  selected  from  an  original  group  of  five,  (2) 
follow-on  development  (referred  to  as  maturity)  and  initial  production 
by  the  winning  contractor,  and  (3)  follow-on  production.  The  acquisition 
strategy  was  designed  to  field  a  system  in  about  6-1/2  years,  from  pro¬ 
gram  initiation  in  February  1977  to  achieving  initial  operational  capa¬ 
bility  in  October  1983 

In  May  1981,  the  Army  awarded  Ford  Aerospace  a  contract  to  complete 
the  weapon’s  development,  which  also  included  three  fixed-price  pro¬ 
duction  options  to  be  exercised  at  1-year  intervals  for  50, 96,  and  130 
fire  units,  respectively,  beginning  in  May  1982.  The  third  year’s  procure¬ 
ment  quantity  was  later  reduced  to  117.  Exercise  of  the  options  was  tied 
to  a  designated  calendar  date  rather  than  to  a  demonstration  of  tech¬ 
nical  performance.  The  Army  exercised  the  first  two  production  options 
in  fiscal  years  1982  and  1983  for  a  total  of  146  units.  Target  prices  for 
these  two  options  were  $350.1  million  and  $438.5  million,  respectively. 

Funds  were  appropriated  for  the  third  production  option  in  fiscal  year 
1984.  Exercise  of  this  option  was  delayed  pending  further  testing 
requested  by  the  Secretary  of  Defense.  Development  tests  of  production 
fire  units  were  completed  in  May  1985.  Operational  tests  of  production 
units  were  completed  in  June  1985.  Evaluations  of  the  results  of  both 
tests  were  furnished  to  the  Secretary  in  late  August.  His  decision  to  ter¬ 
minate  the  program  followed. 


Assessment  of  the 
Acquisition  Strategy 


We  assessed  how  successful  the  Sergeant  York’s  acquisition  strategy 
was  in  containing  cost  growth,  deploying  the  weapon  quickly  and  on 
schedule,  and  producing  a  weapon  system  that  performed  satisfactorily. 
Up  to  the  point  of  termination,  cost  control  appeared  good.  However,  the 
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desired  acceleration  in  fielding  the  Sergeant  York  had  not  been  achieved 
and  the  gun  system’s  performance  was  deficient  in  several  key  areas. 


Cost  Control  Maintained  Up  to  the  program’s  termination,  costs  to  the  government  had  remained 

relatively  stable,  a  significant  achievement  considering  the  pattern  of 
cost  growth  exhibited  by  many  weapons  during  their  development  and 
early  production.  The  use  of  mature  components  contributed  to  this 
favorable  cost  experience  However,  significant  technical  difficulties 
were  encountered  during  attempts  to  integrate  the  components  into  a 
workable  weapon  system.  This  suggests  that  the  use  of  the  fixed-price 
development  contract  and  the  three  annual  fixed-price  production  con¬ 
tract  options  were  more  effective  in  controlling  costs.  The  Sergeant  York 
contract’s  warranty  provisions  also  were  designed  to  contribute  to  fur¬ 
ther  cost  control.  The  fixed-price  options  did  have  a  drawback  in  that 
they  put  pressure  on  decisionmakers  to  proceed  with  production  on 
schedule  despite  the  weapon’s  technical  difficulties  and  unknowns  to 
take  advantage  of  the  favorable  prices  they  offered. 

A  major  challenge  to  the  Army’s  ability  to  maintain  the  program’s  cost 
control  would  have  come  with  the  negotiation  of  the  first  full  production 
contract,  which  was  to  be  awarded  in  January  1986  That  contract  was 
to  be  negotiated  with  the  benefit  of  actual  production  experience  but 
without  the  price  protection  the  fixed  price  options  had  provided.  Can¬ 
cellation  of  the  program’s  third  phase  leaves  unanswered  the  question 
of  whether  cost  control  would  have  continued. 


Accelerated  Deployment  The  desired  acceleration  in  fielding  the  Sergeant  York  was  not  realized. 

Not  Achieved  termination,  the  Army  was  projecting  that  the  Sergeant  York  would 

not  achieve  initial  operational  capability  before  March  1987, 10  years 
after  program  initiation,  compared  with  the  6-1/2  years  originally 
planned.  However,  this  interval  is  still  somewhat  less  than  was  expe¬ 
rienced  by  seven  other  major  Army  weapon  systems  we  selected  for 
comparison.  These  systems,  some  of  the  largest  Army  acquisitions  since 
the  mid-1970’s,  were  acquired  under  more  conventional  strategies  and 
averaged  12  years  from  program  initiation  to  initial  operational  capa¬ 
bility.  Their  acquisition  periods  ranged  from  8  years  for  the  Ml  tank  to 
16  years  for  the  Patriot  missile 

One  reason  for  the  delay  in  fielding  the  Sergeant  York  was  that  the  pro¬ 
totype  gun  systems  the  contractors  delivered  for  testing  were  less  tech¬ 
nically  mature  than  anticipated.  This  caused  testing  delays  and  the  need 
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Serious  Shortcomings  in 
Systenj  Performance 


for  more  testing  than  had  been  planned.  The  integration  of  the  weapon’s 
mjyor  subsystems  and  their  application  to  a  weapon  for  which  they  had 
not  been  originally  designed  apparently  represented  a  greater  technical 
undertaking  than  originally  anticipated  A  second  reason  was  that  in 
some  instances,  the  minimum  government  involvement  and  limited 
testing  features  of  the  acquisition  strategy  prevented  essential  program 
information  from  reaching  decisionmakers  to  support  budget  requests. 
Limited  program  information  regarding  system  performance  and  pro¬ 
gram  progress  contributed  to  the  Army’s  planned  fiscal  year  1981  pur¬ 
chase  of  Sergeant  York  being  deleted  from  that  year’s  defense  budget 
and  resulted  in  a  major  program  delay. 

Both  limited  information  regarding  performance  in  operational  testing 
and  problems  revealed  in  testing  were  key  factors  in  the  Secretary  of 
Defense’s  decision  in  September  1984  to  delete  production  funds  from 
the  fiscal  year  1986  budget  and  defer  further  production  until  the 
results  of  testing  concluded  in  1985  were  available  for  his  review.  The 
wisdom  of  this  decision  was  borne  out  by  the  test  results,  which  showed 
the  Sergeant  York’s  ineffectiveness  in  overcoming  the  threat. 


The  Sergeant  York’s  most  definitive  tests  were  the  initial  production  and 
follow-on  operational  tests  completed  in  mid- 1985.  The  operational  test 
was  the  first  attempt  to  assess  the  Sergeant  York’s  performance  in  a 
realistic  battlefield  environment. 

The  6-month  initial  production  tests  measured  the  weapon’s  perform¬ 
ance  against  the  contract  specifications.  The  Sergeant  York  met  or 
exceeded  141  of  163  specifications  that  were  tested,  but  of  the  22  speci¬ 
fications  not  met,  7  were  considered  by  Army  evaluators  to  be  particu¬ 
larly  significant.  Among  the  seven  were  specifications  which  concerned 
the  reliability  of  the  power  and  actuation  subsystem,  the  gun’s  target 
detection  capabilities,  and  its  survivability  against  certain  countermea¬ 
sures.  Concerning  the  operational  tests,  the  Army’s  assessment  was  that 
the  Sergeant  York  had  performed  significantly  better  than  the  weapon  it 
was  to  replace,  although  it  was  acknowledged  that  there  were  still  some 
shortcomings  to  be  corrected.  However,  according  to  the  evaluation  of 
the  Director,  Operational  Test  and  Evaluation,  in  the  Office  of  the  Secre¬ 
tary  of  Defense — the  one  accepted  by  the  Secretary  of  Defense — the 
Sergeant  York  represented  only  a  marginal  improvement  over  other 
available  air  defense  weapons,  was  ineffective  in  performing  the  air 
defense  mission,  and  was  unsuitable  for  combat  operations  because  it 
had  a  low  operational  availability. 


Page  5 


GAO/NSlAim<89  Sergeant  York 


B'220094 


Protection  Offered  by 
Warranty  Provisions 


The  contract  to  acquire  the  Sergeant  York  provided  protection  against 
cost  increases  emanating  from  defects  in  the  design,  component  integra¬ 
tion,  materials,  or  workmanship  which  could  preclude  meeting  the  speci¬ 
fication  requirements.  This  protection  went  beyond  that  generally 
obtained  for  other  weapon  systems,  where  the  contractor’s  liability  does 
not  cover  design  defects.  The  warranty  provisions  made  the  contractor 
responsible  for  correcting  the  defects  without  increases  to  the  contract’s 
target  cost,  target  ceiling,  or  ceiling  price  and  without  extensions  of 
delivery  time.  Under  the  warranty,  the  performance  of  the  Sergeant 
York  was  to  be  measured  against  the  detailed  requirements  in  the  con¬ 
tract’s  development  specification.  These  requirements  were  to  be  met 
under  the  controlled  conditions  of  a  development  test.  The  initial  pro¬ 
duction  test,  a  development  test  completed  in  May  1985,  was  to  be  the 
basis  for  determining  how  the  Sergeant  York  measured  up  to  these 
requirements.  The  contractor’s  warranty  responsibilities  did  not  extend 
to  performance  of  the  Sergeant  York  in  operational  testing  against  oper¬ 
ational  requirements.  These  requirements,  such  as  mission  performance 
and  survivability,  were  to  be  demonstrated  under  less  controlled  test 
conditions,  which  represented  a  more  realistic  battlefield  environment. 
Such  opierational  testing  was  completed  in  June  1985. 


Conclusions  and 
Recommendations 


The  Sergeant  York’s  acquisition  strategy  contained  some  positive  ele¬ 
ments  but,  on  the  whole,  was  detrimental  to  the  program.  Costs  were 
successfully  controlled  up  to  the  time  the  contract  was  terminated,  and 
the  warranty  provisions  in  the  contract  offered  the  Army  protection 
against  cost  increases  if  modifications  were  needed  to  bring  the  system 
up  to  specifications. 

On  the  other  hand,  the  desired  acceleration  in  fielding  the  Sergeant  York 
was  not  achieved.  This  was  due  both  to  weaknesses  in  the  acquisition 
strategy  and  to  the  technical  performance  difficulties  of  the  weapon 
itself;  however,  it  is  not  possible  to  sort  out  the  relative  contribution  of 
each.  The  technical  difficulties  expenenced  by  the  weapon  system  were 
not  a  fault  of  the  acquisition  strategy.  Technical  difficulties  can  be 
encountered  in  developing  any  complex  system,  irrespective  of  the 
acquisition  strategy  used.  However,  the  Sergeant  York’s  tight  schedule 
and  the  limited  operational  testing,  both  of  which  were  critical  elements 
of  the  strategy,  left  few  opportunities  to  resolve  these  difficulties  before 
major  production  commitments  were  made.  Also,  while  they  were  bene¬ 
ficial  from  a  cost  control  standpoint,  the  fixed-price  production  options 
put  more  pressure  on  decisionmakers  to  proceed  on  schedule  even 
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though  they  recognized  the  risks  the  weapon’s  technical  difficulties 
posed. 

Another  disadvantage  of  the  strategy  was  that  because  it  limited  the 
opportunities  for  testing,  it  did  not  give  decisionmakers  as  much  infor¬ 
mation  at  given  points  in  time  as  they  normally  get  under  more  conven¬ 
tional  strategies.  In  this  sense,  the  strategy  did  not  provide  essential 
information  regarding  the  Sergeant  York’s  performance  when  needed  to 
support  budget  decisions.  The  need  for  such  information  led  to  funding 
delays.  Concern  over  the  lack  of  operational  testing  became  a  key  stum¬ 
bling  block  to  the  program’s  future  but  not  until  after  the  system  was 
well  into  production.  The  deferral  of  some  tests  and  the  mixed  results  of 
others  made  the  strategy  to  limit  operational  testing  no  longer 
acceptable. 

Confining  surveillance  over  the  Sergeant  York’s  development  early  in 
the  program  to  just  a  few  individuals  within  the  Army  provided  little 
opportunity  to  hold  the  program  up  to  wider  and  more  critical  scrutiny 
in  the  Army’s  higher  echelons,  the  Office  of  the  Secretary  of  Defense, 
and  the  Congress.  This,  coupled  with  confidence  in  the  use  of  proven 
components,  fueled  optimism  about  the  Sergeant  York’s  eventual  per¬ 
formance  and  enabled  the  initial  production  decision  to  be  made  on  the 
basis  of  limited  testing.  Subsequent  test  results  showed  these  views  to  be 
overly  optimistic. 

The  program  also  provides  some  insight  into  the  limitations  of  a  con¬ 
tractor’s  warranty.  The  Sergeant  York  warranty,  while  it  provided  pro¬ 
tection  against  the  weapon  system  not  meeting  the  contract’s 
performance  specifications,  offered  no  guarantee  that  the  weapon  would 
perform  satisfactorily  in  an  operational  environment,  where  factors 
such  as  weather,  terrain,  and  countermeasures  aggravate  the  difficulties 
of  coping  with  the  threat.  However,  the  benefits  derived  from  having  a 
warranty  which  requires  that  a  system  perform  to  specifications  in  a 
nonoperational  environment  probably  will,  to  a  certain  extent,  carry 
over  to  its  performance  in  an  operational  environment. 

The  Army  should  continue  trying  to  reduce  the  time  it  takes  to  develop 
and  field  new  weapons  as  long  as  performance  is  not  adversely  affected. 
However,  we  believe  there  are  lessons  to  be  learned  from  the  events  that 
followed  the  application  of  the  accelerated  acquisition  strategy  to  the 
Sergeant  York  program  and  are  recommending  that,  when  the  Army 
considers  such  strategies  for  future  programs,  the  Secretary  of  Defense 
require  the  Army  to: 
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Make  an  initial  detailed  assessment  of  the  technical  risks  involved  in 
individual  subsystems,  as  well  as  in  the  integration  of  those  subsystems 
into  one  system,  with  an  explicit  focus  on  whether  the  technology  being 
attempted  is  compatible  with  an  accelerated  acquisition  strategy.  Built 
into  the  strategy  should  be  provisions  for  adjusting  schedules  and  other 
program  facets  if  technical  difficulties  occur. 

Assess  the  weapon’s  technological  progress  periodically  to  see  if  it  is  still 
compatible  with  the  planned  acceleration.  If  technical  progress  is  no 
longer  keeping  pace  with  the  acceleration,  the  strategy  should  be 
adjusted  to  bring  it  in  line  with  the  technology. 

Ensure  that  the  strategy  provides  for  testing  and  evaluation  sufficient 
to  identify  for  decisionmakers  any  major  shortcomings  in  a  weapon’s 
operational  suitability  and  effectiveness  which  have  to  be  resolved 
before  initial  production  is  approved. 


DOD  generally  concurred  in  our  findings  and  recommendations.  It 
expressed  the  view  that  there  was  no  need  for  further  guidance  on 
assessing  risks  and  on  assessing  the  continuing  compatibility  of  a 
weapon  system  with  its  acquisition  strategy.  The  Department  noted  that 
such  guidance  was  in  effect  when  the  Sergeant  York’s  acquisition 
strategy  was  devised.  In  the  case  of  the  Sergeant  York,  however,  dod 
said  a  conscious  decision  had  been  made  to  use  a  unique  acquisition 
strategy  with  attendant  recognized  risks.  It  recognized  that  specific  les¬ 
sons  had  been  learned,  particularly  with  respect  to  the  “hands  off’ 
aspects  of  the  strategy.  It  believed  procedures  in  its  departmental  direc¬ 
tives  and  lessons  learned  from  the  Sergeant  York’s  acquisition  should 
help  preclude  a  recurrence  of  the  types  of  difficulties  that  were  expe¬ 
rienced  with  the  Sergeant  York. 

Although  additional  guidance  may  not  be  necessary,  we  believe  atten¬ 
tion  must  be  given  to  enforcing  existing  guidance  in  accelerated  acquisi¬ 
tion  strategies  to  help  ensure  (1)  early  recognition  of  indications  that 
technical  risks  may  be  present  which  are  greater  than  originally  antici¬ 
pated  and  (2)  adjustments  to  strategies,  as  necessary,  to  bring  them  into 
line  with  the  technical  risks  before  programs  have  progressed  too  far. 

While  agreeing  with  the  need  to  furnish  decisionmakers  with  evalua¬ 
tions  of  weapon  system  performance  based  on  adequate,  realistic  opera¬ 
tional  testing,  DOD  noted  that  emphasis  on  such  testing  had  been 
incorporated  in  a  revised  DOD  Directive  6000.3,  published  in  March  1986. 
According  to  dod,  this  directive  incorporates  the  requirements  of  Public 
Law  98-94,  dated  September  24,  1983,  which  established  the  Office  of 
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Operational  Test  and  Evaluation  under  the  Secretary  of  Defense.  That 
law  stipulates  that  a  full  production  decision  shall  not  be  made  until  the 
director  of  that  office  has  submitted  a  report  to  the  Secretary  of  Defense 
and  to  the  Congress  stating  that  adequate  tests  of  the  system  have  been 
conducted  and  the  results  have  confirmed  that  it  is  effective  and  suit¬ 
able  for  combat.  The  Department  noted  that  the  decision  to  produce  the 
Sergeant  York  had  been  made  before  the  law  was  passed 

The  proper  implementation  of  Public  Law  98-94  should  ensure  an  evalu¬ 
ation  of  operational  test  results  before  a  full  production  decision  is 
made.  There  are  cases  when  similar  caution  should  be  observed  before  a 
system  is  allowed  to  begin  initial  low-rate  production.  Our  concern  stems 
from  the  fact  that  the  Sergeant  York,  in  accordance  with  its  accelerated 
acquisition  strategy,  began  initial  production  without  a  confirmation 
that  the  weapon  could  fulfill  its  basic  mission  in  an  operational  environ¬ 
ment — protecting  friendly  forces  against  enemy  helicopters.  In  acceler¬ 
ated  programs,  such  as  the  Sergeant  York,  there  is  a  danger  that 
operational  testing  requirements  will  be  relaxed  without  provisions 
being  made  in  the  acquisition  strategy  that  would  enable  identifying  for 
decisionmakers  any  major  shortcomings  in  operational  effectiveness 
prior  to  initiating  production. 

Our  draft  report  included  a  recommendation  on  this  matter,  stated  in 
general  terms,  which  called  for  evaluations  of  performance  based  on 
adequate  and  realistic  operational  testing  to  be  made  before  the  produc¬ 
tion  decision.  Considering  that  operational  testing  requirements  of 
Public  Law  98-94  have  been  incorporated  into  revised  DOD  Directive 
5000.3,  we  have  modified  our  recommendation  to  focus  on  the  need  for 
an  accelerated  acquisition  strategy  to  provide  for  an  identification  of 
major  shortcomings  in  the  weapon’s  operational  effectiveness  and  suita¬ 
bility  that  must  be  resolved  before  it  is  approved  for  initial  production. 

DOD  also  suggested  other  changes  in  the  report  which  we  have  incorpo¬ 
rated,  as  appropriate. 
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The  details  of  our  findings  appear  in  appendix  I.  dod’s  comments  are  in 
appendix  II. 

As  arranged  with  your  office,  unless  you  publicly  announce  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  3  days  from 
the  date  of  the  report.  At  that  time  we  will  send  copies  to  interested 
parties  and  make  copies  available  to  others  upon  request. 

Sincerely  yours. 


Frank  C.  Conahan 
Director 


I 
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Assessment  of  the  Accelerated  Strategy  Used 
to  Acquire  the  Sergeant  York 


Introduction 


The  Army  plarmed  to  acquire  the  Sergeant  York  under  an  accelerated 
strategy.  As  originally  conceived,  this  approach  contained  features  such 
as  concurrent  development  and  production  and  limited  testing,  coupled 
with  the  use  of  components  that  had  already  been  developed  for  other 
weapon  systems.  Together,  these  features  were  expected  to  enable  the 
Army  to  field  the  gun  several  years  sooner  than  it  takes  to  field 
weapons  acquired  under  more  conventional  programs  and  to  do  so  at 
substantial  savings. 

The  original  acquisition  strategy,  dating  back  to  February  1977,  was  to 
be  accomplished  in  three  phases  containing  several  interrelated  ele¬ 
ments.  These  consisted  of  engineering  development,  maturity  and  initial 
production,  and  follow-on  production.  The  first  phase  was  to  include  a 
29-month  competitive  development  effort  between  two  contractors, 
selected  from  an  original  group  of  five,  culminating  in  a  “shootoff  ’  to  be 
followed  by  source  selection.  In  the  second  phase,  the  selected  gun 
system  would  complete  development  during  a  maturity  phase,  which 
was  to  involve  correcting  problems  discovered  in  the  competitive  test 
and  conducting  additional  tests  to  ascertain  whether  weapon  system 
performance  had  improved.  Initial  production  was  to  begin  in  phase  II 
as  well.  Phase  III  encompassed  producing  the  remaining  gun  systems 
and  achieving  initial  operational  capability. 

The  Army  planned  to  buy  614  units  of  the  Sergeant  York.  However,  on 
August  27, 1986,  the  Secretary  of  Defense  terminated  the  program  after 
only  64  units  had  been  delivered.  The  Secretary  cited  the  weapon’s 
unsatisfactory  performance  in  recent  tests  which  Department  of 
Defense  (dod)  test  evaluators  said  showed  little  improvement  over 
existing  air  defense  systems.  The  Secretary  felt  that  completing  the  pro¬ 
gram  was  not  worth  the  investment. 


Features  of  the 
Sergeant  York 
Acquisition  Strategy 


The  Sergeant  York’s  acquisition  strategy  was  unique  in  that  it  combined 
several  features  in  an  attempt  to  accelerate  fielding.  These  features 
included  parallel,  competitive  development;  a  “skunk  works’’  approach 
to  contracting  in  which  contractors  were  given  free  rein  to  design  the 
system  in  a  way  they  thought  most  appropriate  for  meeting  the  Army’s 
requirements;  a  “hands-off’  approach  in  which  the  government’s  man¬ 
agement  role  was  minimal  during  the  weapon’s  development;  a  short, 
combined  development  and  operational  test  prior  to  production;  and  a 
phase  which  combined  the  system’s  follow-on  development,  referred  to 
as  the  maturity  phase,  and  initial  production.  These  factors  were 
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intended  to  significantly  shorten  the  development  phases  preceding  ini¬ 
tial  production.  The  accelerated  program  was  recognized  as  having  tech¬ 
nical  and  cost  risks.  Therefore,  the  program  included  several  safeguards 
that  the  Army  believed  would  minimize  these  risks.  Prominent  among 
these  were  the  use  of  available,  proven  components  and  the  negotiation 
of  (1)  a  fixed-price  development  contract  with  fixed-price  production 
options  and  (2)  warranty  provisions  intended  to  protect  the  government 
against  cost  increases  resulting  from  corrections  of  design  and  other 
deficiencies. 


Competitive  Prototyping  The  first  phase  of  the  acquisition  strategy  was  to  be  a  development  com¬ 
petition  between  two  contractors.  This  was  to  culminate  in  each  con¬ 
tractor  delivering  complete  prototypes  to  the  Army  for  testing.  The 
advantages  of  this  approach  were  having  more  than  one  candidate  for 
the  mission,  having  actual  hardware  on  which  to  base  a  comparison,  and 
stimulating  individual  contractor  creativity  in  devising  a  weapon  to  ful¬ 
fill  the  mission. 

Some  of  the  advantages  of  the  competition  may  have  been  lost  when  the 
Army  decided  it  would  select  a  contractor  for  the  phase  II  full-scale 
development  and  initial  production  contract  after  the  competitive  devel¬ 
opment  and  operational  tests  were  completed  and  evaluated.  Both  the 
Ford  Aerospace  prototype,  the  one  selected,  and  the  competing  proto¬ 
type  submitted  by  the  General  Dynamics  Corporation  displayed 
numerous  shortcomings  during  the  tests.  It  was  evident  at  the  time  that 
both  prototypes  still  required  considerable  development.  This  is  not  to 
question  the  Army’s  preference  for  the  Ford  Aerospace  prototype  at 
I  that  stage  of  development  but  merely  to  point  out  that  in  view  of  the 

•  immaturity  of  the  two  prototypes,  it  may  have  been  prudent  to  wait  for 

the  further  development  of  both  prototyjies  before  making  a  selection. 

We  had  recommended  such  a  course  in  a  report  to  the  Congress  in  Jan¬ 
uary  1980.'  In  that  report,  we  pointed  out  that  important  durability  and 
maintainability  tests  would  not  be  conducted  until  after  the  source  selec¬ 
tion.  DOD  responded  that  (1)  durability  tests  were  not  expected  to  affect 
the  competition  since  Army  engineering  judgment  indicated  that  the 
results  were  likely  to  be  similar  for  both  prototypes,  (2)  the  cost  and 
time  to  run  tests  which  were  not  expected  to  affect  the  competition  were 


‘Inherent  Risk  m  the  Army's  Acquisition  Strategy  Demands  Particular  Caution  in  Evaluating  the 
Division  Air  Defense  Gun  System’s  Production  Readiness  (GAO/C-PSAD-80-9,  Jan  31,1980) 
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needed  elsewhere,  and  (3)  the  competition  was  being  driven  by  other 
tests  which  the  Army  would  conduct. 


Minimum  Government 
Invo  vement  and  Contractor 
Flexibility 


The  Sergeant  York  acquisition  strategy  featured  a  “hands-off,  skunk 
works”  approach  to  the  weapon’s  development.  This  emphasized  min¬ 
imum  government  management  during  development  and  limited  the 
Army’s  activities  to  holding  quarterly  2-day  design  review  meetings  at 
each  contractor’s  location  and  to  reviewing  contractor-generated 
reports.  The  strategy  embodied  having  thinly  staffed  government  and 
contractor  project  offices  shielded  from  interference  by  other  concerned 
government  officials,  reducing  the  reporting  requirements,  limiting 
access  to  contractors  and  contractor  information  considered  to  be  com¬ 
petition  sensitive,  and  giving  the  contractors  the  freedom  and  responsi¬ 
bility  to  design  and  develop  a  weapon  that  would  satisfy  Army 
requirements. 


Critical  to  this  flexibility  was  the  authority  each  contractor  was  given  to 
trade  off  some  of  the  requirements  for  others  on  the  basis  of  perform¬ 
ance  versus  costs.  The  Army  established  12  firm  requirements  that  each 
competing  contractor’s  weapon  system  had  to  meet.  Beyond  these,  the 
Army  identified  43  system  requirements  in  priority  order  which  each 
contractor  could  trade  off  to  help  lower  the  program’s  cost.  For 
example.  Ford  Aerospace  elected  not  to  equip  its  weapon  with  night 
vision  capability,  1  of  the  43  tradable  items,  in  an  effort  to  keep  unit 
production  costs  down. 


Concurrency  and  Limited 
Testing  • 


To  enhance  the  prospects  of  accelerating  the  Sergeant  York’s  fielding, 
the  Army  chose  to  (1)  perform  developmental  activities  concurrently 
with  production  and  (2)  conduct  less  testing  before  the  production  deci¬ 
sion  than  is  usually  conducted  under  a  more  conventional  acquisition 
strategy. 


Under  the  strategy,  the  competitive  “shootoff”  test  was  the  only  govern¬ 
ment  test  scheduled  before  the  Army  was  to  select  a  source  and  award 
the  phase  II  contract.  Phase  II  itself  was  to  involve  initiating  production 
before  completing  the  development  of  the  weapon  system.  Another  gov¬ 
ernment  test,  referred  to  as  a  “check  test,”  was  planned  at  the  conclu¬ 
sion  of  the  maturity  phase  to  ascertain  whether  the  winning  contractor 
had  resolved  problems  revealed  in  its  system  during  the  competitive 
testing.  The  next  government  test  was  to  involve  initial  production  units 
and  was  not  slated  to  begin  until  the  end  of  phase  II. 
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As  a  consequence  of  the  desire  to  accelerate  the  schedule  and  to  mam- 
tain  minimum  government  involvement,  substantial  portions  of  develop¬ 
ment  and  operational  testing  were  not  planned  to  be  held  until  after 
production  was  to  begin.  Development  tests  deferred  until  after  produc¬ 
tion  began  included  operation  in  a  cold  climate,  durability  of  the  system 
in  stressful  battlefield  conditions,  system  transportability,  and  electro¬ 
magnetic  compatibility  and  interferende. 

Although  certain  government  operational  tests  were  to  be  held  before 
the  production  decision,  they  were  constrained  by  the  limited  realism  of 
the  test  environment,  which  to  some  extent  was  unavoidable  because  of 
safety  considerations;  the  deferral  of  some  activities  usually  begun 
during  development,  such  as  integrated  logistic  support;  and  the  few  fire 
units  made  available  for  testing.^  Oiierational  issues  which  were  not 
planned  to  be  fully  evaluated  before  the  production  decision  were  opera¬ 
tion  and  maintenance  by  Army  personnel;  proposed  doctrine,  tactics, 
and  training;  and  system  effectiveness  and  survivability  as  part  of  a 
larger  force  in  combat. 

Development  of  integrated  logistic  support  requirements  and  resources 
was  deferred  until  after  the  winning  contractor  was  selected  so  that 
these  costs  would  not  be  incurred  to  support  more  than  one  system.  Inte¬ 
grated  logistic  support  includes  the  concept  for  maintaining  the  weapon 
in  the  field,  procuring  test  and  support  equipment,  training  personnel, 
and  preparing  operating  and  maintenance  manuals.  Development  of 
these  elements  is  important  to  operational  testing,  for  a  key  element  of 
such  testing  is  the  Army’s  ability  to  train  soldiers  to  operate  and  main¬ 
tain  the  weapon  under  battlefield  conditions. 


Safeguards  Against 
Risk 


The  Sergeant  York’s  accelerated  acquisition  strategy  involved  risks,  par¬ 
ticularly  in  view  of  the  planned  concurrency  and  limited  testing.  To 
make  these  risks  acceptable,  several  safeguards  were  built  into  the  pro¬ 
gram.  These  included  using  proven  components,  obtaining  more  exten¬ 
sive  warranty  coverage  than  generally  found  in  other  major  weapon 
contracts,  and  negotiating  fixed-price  development  and  production 
contracts. 


^Operational  testing  is  intended  to  assess  system  operational  effectiveness  and  reliability,  when  oper- 
at^  and  maintained  by  military  personnel,  m  a  re^istic  combat  environment  and  against  a  simulated 
enemy  threat  Operational  effectiveness  is  the  capability  of  the  weapon  to  accomplish  its  mission 
Operational  suitability  is  the  weapon’s  capability  to  be  satisfactorily  placed  m  field  use  considering, 
among  other  thmgs,  the  capability  to  operate,  mamtam,  and  support  the  weapon 
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Use  of  Proven  Components  A  ground  rule  established  for  the  phase  I  competition  was  that  each  con¬ 
tractor  was  to  use  already  proven  subsystems  in  weapon  designs.  It  was 
felt  that  use  of  proven  subsystems  rather  than  an  all-new  design  would 
reduce  potential  design  problems  and  allow  for  an  accelerated  develop¬ 
ment  schedule.  The  major  development  task  was  anticipated  to  be  inte¬ 
gration  of  the  subsystems.  Each  contractor  was  to  use  the  M48A6  tank 
chassis  as  the  vehicle  for  its  weapon  system,  and  each  chose  a  mature 
radar  and  gun  system  to  include  in  its  design. 


Warranty  Provisions  When  Ford  Aerospace  emerged  as  the  winner  of  the  phase  I  competition, 

the  contractor  accepted  responsibility  for  the  weapon  system’s  meeting 
the  technical  specifications  in  the  maturity  phase.  These  specifications 
were  derived  from  the  Army’s  stated  air  defense  gun  requirements.  In 
essence,  the  warranty  required  Ford  Aerospace  to  correct  characteris¬ 
tics  of  the  weapon  system  which  did  not  meet  the  specifications  at  no 
increase  in  price  to  the  Army.  The  contract  also  required  Ford  Aero¬ 
space  to  correct  problems  attributable  to  defective  workmanship  or 
material.  In  the  Army’s  view,  this  warranty  represented  a  safeguard 
against  cost  increases  that  could  arise  from  a  need  to  correct  perform¬ 
ance  deficiencies. 

I 

According  to  the  Army,  prior  to  the  warranty  legislation  included  as  sec¬ 
tion  794  of  the  fiscal  year  1984  Department  of  Defense  Appropriation 
Act,  the  standard  warranty  clause  in  procurement  regulations  tradition¬ 
ally  covered  only  workmanship  defects.  By  obtaining  a  warranty  against 
design  and  integration  deficiencies,  the  Army  obtained  protection 
against  claims  arising  from  design  defects  which  are  typically  the  more 
I  significant  problem  for  the  government. 

Section  794  required  the  government,  in  weapon  system  production  con¬ 
tracts,  to  obtain  written  guarantees,  or  warranties,  from  the  prime  con- 

'  tractor.  The  warranties  were  to  (1)  cover  conformance  of  the  design  and 

manufacture  of  the  system  and  its  components  to  the  government’s  spe¬ 
cifically  delineated  performance  requirements  and  (2)  ensure  that  the 
system  and  its  components  were  free  of  defects  in  materials  and  work¬ 
manship  which  would  cause  the  system  to  fail  to  meet  the  government’s 
performance  requirements. 

The  contract  for  initial  production  of  the  Sergeant  York  was  awarded  in 
May  1981,  before  enactment  of  section  794.  Nevertheless,  the  contract’s 
correction  of  deficiencies  and  system  responsibility  provisions  parallel 
provisions  found  in  section  794  in  the  areas  of 
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• 

• 

• 

design  and  manufacturing  deficiencies, 

deficiencies  in  materials  and  workmanship,  and 

costs  to  repair  or  replace  parts  to  achieve  required  performance. 

Fixed-Price  Contracts 

Fixed-price  development  contracts  were  awarded  to  the  two  contractors 
that  participated  in  the  competition.  In  addition  to  offering  cost  control, 
this  type  of  contract  was  viewed  as  complementing  the  “hands-off’  and 
tradable  requirements  features  by  eliciting  the  contractors’  best  efforts 
for  a  given  price.  The  three  production  options  included  in  phase  II  were 
also  fixed  priced  (with  incentives).  Each  contractor  made  proposals  on 
these  production  options  during  the  competition.  The  intent  was  to  pro¬ 
vide  protection  against  cost  growth  during  production  when  only  one 
contractor  would  be  producing  the  weapon.  The  production  prices  thus 
had  the  benefit  of  being  established  through  competition.  The  Army  also 
held  open  the  possibility  of  reopening  competition  for  phase  III  produc¬ 
tion  as  additional  cost  protection. 

Program  Results 

! 

I 

In  our  1983  report  on  the  Sergeant  York,^  we  noted  that  in  the  final  anal¬ 
ysis,  the  acquisition  strategy’s  success  would  be  measured  by  its  contri¬ 
bution  to  the  Army’s  ability  to  (1)  contain  cost  growth,  (2)  deploy  the 
weapon  on  schedule,  and  (3)  produce  an  effective  system.  Up  to  the  time 
the  program  was  canceled,  cost  control  appeared  good.  However,  it  was 
evident  that  the  desired  acceleration  in  fielding  the  Sergeant  York  would 
not  be  achieved  and  that  technical  performance  as  measured  by  test 
results  was  inadequate  to  meet  the  threat. 

1 

The  contractual  aspects  of  the  strategy  did  hold  down  program  costs. 
However,  this  advantage  was  outweighed  by  other  elements  of  the 
strategy,  which  contributed  to  the  program’s  cancellation.  These  were 
(1)  the  “hands  off’  approach  during  the  development  phase,  which  lim¬ 
ited  program  reviews  of  the  contractor’s  progress  to  only  a  few  individ¬ 
uals  within  the  Army  and  inhibited  wider  dissemination  of  information 
about  the  program,  and  (2)  the  reduced  testing  that  the  strategy 
dictated. 

Cost  Growth  Was  Contained 

Costs  to  the  government  for  the  Sergeant  York  remained  relatively 
stable.  The  fact  that  program  costs  did  not  increase  significantly  was 

^The  Army  Should  Confirm  Sergeant  York  Air  Defense  Gun’s  Reliability  and  Maintainability  Before 
Exercisinit  Next  Production  Option  (GAO/MASAD-83-8.  Jan  27, 1983) 
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commendable  given  the  cost  growth  other  Army  weapons  have  expe¬ 
rienced,  particularly  as  they  make  the  transition  from  development  into 
production. 

The  control  of  costs  can  be  attributed  to  three  features  of  the  Sergeant 
York’s  acquisition  strategy:  (1)  the  use  of  fixed-price  production  options 
covering  nearly  half  the  planned  quantities,  (2)  the  use  of  proven  com¬ 
ponents  to  reduce  the  technical  development  costs,  and  (3)  the  use  of 
integrated  prototypes  during  competition.  Considering  that  despite 
using  proven  components  and  integrated  prototypes,  the  program  still 
experienced  significant  difficulties  with  integrating  components  and 
encountered  serious  performance  problems,  the  use  of  fixed-price  con¬ 
tracts  emerges  as  the  most  effective  cost  control  feature  of  the  Sergeant 
York’s  acquisition  strategy.  Another  feature  of  the  strategy  which  the 
Army  expected  to  contribute  to  cost  control  was  the  broadened  protec¬ 
tion  offered  by  the  contract’s  warranty  provisions,  which  covered  not 
only  deficiencies  in  materials  and  workmanship  but  design  deficiencies 
as  well.  The  contractor  had  to  correct  design  deficiencies  at  its  expense 
for  the  system  to  meet  specifications. 

Up  to  the  time  the  program  was  canceled,  it  was  evident  that  the  acqui¬ 
sition  strategy  was  helpful  in  controlling  costs.  However,  the  program 
never  reached  the  point  where  the  Army  would  face  the  challenge  of 
negotiating  the  next  production  contract  based  on  actual  production 
experience.  In  the  past,  several  weapon  systems  have  experienced  sig¬ 
nificant  cost  growth  at  this  point  as  the  actual  costs  on  the  previous 
production  contracts  were  higher  than  expected  and  became  the  basis 
for  negotiating  higher  prices  on  new  contracts. 

While  the  fixed  price  options  must  be  credited  with  holding  down  the 
cost  of  the  first  146  gun  systems  under  contract,  the  options  did  have  a 
drawback  in  that  officials  gave  too  much  weight  to  them  when  consid¬ 
ering  decisions  whether  to  proceed  with  production  on  schedule  even 
when  serious  technical  difficulties  were  present  and  critical  perform¬ 
ance  capabilities  were  not  demonstrated.  In  responses  to  our  prior 
reports  in  which  we  urged  caution  in  proceeding  with  Sergeant  York’s 
production,  the  Department  of  Defense  maintained  that  technical  risks 
were  not  so  great  as  to  warrant  delays  and  that  cost  savings  would 
result  by  maintaining  the  schedule.  Army  officials  maintained  that 
deferring  production  would  cause  a  lost  opportunity  to  take  advantage 
of  favorable  procurement  options. 
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The  acquisition  strategy  did  not  achieve  early  deployment  of  the  Ser¬ 
geant  York,  one  of  its  major  objectives.  The  gains  in  time  the  Army 
anticipated  from  combining  development  and  initial  production  were 
never  realized.  Starting  with  the  immaturity  of  the  prototypes  delivered 
for  the  1980  competitive  tests,  the  program  began  to  slip.  Tests  were 
delayed,  others  had  to  be  added,  and  the  Office  of  the  Secretary  of 
Defense  became  involved  in  reviewing  the  program  and  in  evaluating 
the  Sergeant  York’s  performance  in  the  tests. 

Shown  in  table  I.l  is  the  latest  schedule  that  the  Army  projected  for  the 
Sergeant  York  compared  with  the  original  schedule.  Dates  are  not  shown 
in  the  original  schedule  for  tests  which  were  added  later. 
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Table  1.1  Actual  or  Projoctad  Schadula 
of  Evanta  Comparad  With  Original 
Schadula 


Actual  or 
prolected  schedule 

Original 

schedule 

Phase  1  (Engineering  Development) 

Award  competitive  engineering  development  contracts 

Jan  1978 

Jan  1978 

Competitive  development  and  operational  test 

start 

June  1980 

June  1980 

Complete 

Nov  1980 

Sept  1980 

Phase  II  (Maturity  and  Initial  Production) 

Award  phase  II  contract 

May  1981 

Oct  1980 

Check  test 

Start 

Nov  1981 

Oct  1980 

Complete 

Jan  1982 

Dec  1980 

Production  decision 

May  1982 

Oct  1980 

Reliability,  availability,  maintainability,  durability  test 

start 

Feb  1982 

Complete 

Apr  1982 

Contractor  engineering  test 

Start 

Apr  1982 

Complete 

Dec  1982 

Engineering  prototype  unit  test 

Start 

May  1983 

Complete 

Aug  1983 

Deliver  first  production  unit 

Mar  1984 

June  1982 

Design  verification  test 

Start 

Mar  1984 

Complete 

Dec  1984 

Limited  test 

start 

July  1984 

Complete 

Aug  1984 

Initial  production  test 

Start 

Dec  1984 

July  1982 

Complete 

May  1985 

Jan  1983 

Follow-on-evaluation  1 

Start 

Apr  1985 

Complete 

June  1985 

Phase  III  (Follow-on  Production) 

Award  phase  III  contract 

Jan  1986 

Oct  1982 

Initial  operational  capability 

Mar  1987 

Oct  1983 

Follow-on  evaluation  II 

start 

Feb  1987 

Complete 

Apr  1987 

Production  complete 

June  1990 

Aug  1987 
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According  to  the  actual/projected  schedule,  initial  operational  capability 
of  the  Sergeant  York,  had  the  program  continued,  would  have  been 
achieved  about  3-1/2  years  later  than  called  for  under  the  original 
schedule.  Whereas  initial  operational  capability  was  to  be  achieved  in 
about  6-1/2  years  after  program  initiation  in  February  1977,  the  Army 
was  projecting  it  would  take  about  10  years  to  achieve.  The  major 
causes  of  this  delay  appear  to  be  (1)  deletion  of  fiscal  year  1981  produc¬ 
tion  due  to  funding  cuts  which  delayed  the  first  year’s  production  by 
about  1-1/2  years,  (2)  production  start-up  problems,  which  delayed 
deliveries,  and  (3)  deferral  of  the  third  production  option  decision  to 
conduct  additional  operational  testing,  which  led  to  stretching  out  deliv¬ 
eries  of  units  already  under  contract  to  avoid  a  production  line 
shutdown. 

To  compare  the  time  it  took  to  begin  the  planned  fielding  of  the  Sergeant 
York  with  the  time  it  took  to  field  weapon  systems  acquired  under  more 
conventional  strategies,  we  selected  seven  systems  representing  some  of 
the  largest  Army  weapon  programs  fielded  since  the  mid-1970’s.  These 
averaged  nearly  12  years  from  program  initiation  to  initial  operational 
capability.  Thus,  the  Sergeant  York’s  latest  schedule  still  represented 
quicker  fielding  than  the  average  time  for  the  seven  systems  using  more 
conventional  strategies.  The  systems  are  shown  in  table  1.2. 


TaU*  1.2  Acquisition  Time  Frames  for 
Other  Army  Weapons 


Weapon  System 

Years  to 
Field 

AH-64  helicopter 

135 

Copperhead  projectile 

108 

Hellfire  missile 

133 

M1  tank 

82 

Bradley  Fighting  Vehicle 

11  7 

Patriot  missile 

157 

Pershing  II  missile 

99 

On  the  other  hand,  the  Sergeant  York’s  schedule  was  not  as  rapid  as  two 
major  weapons  already  being  fielded  under  accelerated  strategies — the 
Multiple  Launch  Rocket  System  and  the  Black  Hawk  helicopter — which 
achieved  initial  operational  capability  6.2  years  and  8.4  years,  respec¬ 
tively,  after  program  initiation.  It  is  also  interesting  to  note  that  while 
the  l^rgeant  York  saved  time  by  essentially  bypassing  the  advanced 
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Comparison  With  the  Multiple 
launch  Rocket  System  Acquisition 


development  phase,  it  was  destined  to  spend  more  time  between  the  ini¬ 
tiation  of  engineering  development  and  the  latest  projected  initial  opera¬ 
tional  capability  (9.3  years)  than  the  seven  weapons  acquired  under 
conventional  strategies  (an  average  of  7.7  years). 


The  Army  employed  an  acquisition  strategy  similar  to  the  Sergeant 
York’s  to  acquire  the  Multiple  Launch  Rocket  System.  It  consisted  of  a 
competitive  development  phase  of  29  months  (later  extended  to  32 
months),  which  concluded  with  a  combined  development  and  opera¬ 
tional  test.  On  this  basis,  a  contract  was  awarded  to  a  sole  source  for  a 
maturity  phase  and  low-rate  production  with  options  for  additional  low- 
rate  production.  Initial  operational  capability  of  the  system  was 
achieved  in  March  1983,  just  over  6  years  after  program  initiation  and  4 
months  later  than  planned. 

Based  on  our  limited  review  of  the  Multiple  Launch  Rocket  System,  sev¬ 
eral  factors  appear  to  explain  why,  under  a  similar  strategy,  the  system 
was  fielded  essentially  within  6  years. 

First,  the  Multiple  Launch  Rocket  System  is  made  up  of  previously 
applied  technology,  as  was  the  Sergeant  York.  However,  Army  docu¬ 
ments  show  that  the  weapon  design  is  relatively  uncomplicated  and  no 
gaps  existed  in  its  technology.  Being  akin  to  an  artillery  piece,  it  does 
not  have  to  acquire  individual  targets,  such  as  tanks  or  aircraft,  nor 
does  it  engage  moving  targets.  Therefore,  it  does  not  have  a  radar,  has 
less  sophisticated  fire  control,  and  does  not  have  to  fight  at  the  front  of 
the  battle  alongside  tanks.  These  factors  make  for  a  less  sophisticated 
weapon  than  the  Sergeant  York  was  to  be. 

Second,  time  was  set  aside  in  the  Multiple  Launch  Rocket  System  acqui¬ 
sition  strategy  for  final  operational  testing  of  production  units  prior  to 
the  full-scale  production  decision.  The  Sergeant  York  strategy,  as  origi¬ 
nally  conceived,  did  not  provide  for  such  testing  until  after  full-scale 
production  had  been  approved. 

Third,  although  the  rocket  system  exhibited  reliability  and  accuracy 
problems  even  when  full-scale  production  began,  the  program  was  not 
delayed  to  effect  solutions,  as  was  done  with  the  Sergeant  York.  We  did 
not  determine  why  technical  problems  had  been  handled  differently  in 
the  Multiple  Launch  Rocket  System,  but  two  factors  may  have  been 
influential.  First,  reliability  was  only  1  percent  below  minimum  accept¬ 
able  values.  Second,  the  fact  that  a  second  operational  test  had  been 


Page  24 


6AO/NSIAI>86-89  Sergeant  York 


Appendix  1 

AMessment  of  the  Accelerated  Strategy  Used 
to  Acquire  the  Sergeant  York 


Strategy  Underestimated  Need  for 
Testing 


I 


I 


conducted  on  the  system  may  have  given  decisionmakers  more  confi¬ 
dence  in  the  weapon’s  performance. 


Sergeant  York  test  units  were  less  technically  mature  than  anticipated 
during  the  program,  giving  rise  to  testing  delays  and  the  need  for  addi¬ 
tional  testing.  Apparently,  the  integration  of  the  weapon’s  major  subsys¬ 
tems,  as  well  as  their  application  to  a  weapon  for  which  they  had  not 
been  originally  designed,  represented  a  greater  technical  undertaking 
than  anticipated.  Evaluations  of  the  Sergeant  York’s  test  performance 
by  the  Office  of  the  Secretary  of  Defense  were  less  optimistic  than  the 
Army’s  and  led  to  changes  in  the  program’s  course.  Under  pressure  to 
demonstrate  the  weapon’s  capability  before  it  could  obtain  approval  to 
proceed  with  the  third  year’s  procurement,  the  Army  added  some  tests 
and  moved  others  forward. 

Under  the  acquisition  strategy,  following  a  30-day  demonstration  of 
each  contractor’s  prototype,  the  two  competing  contractors  were  to 
deliver  prototypes  to  the  Army  for  the  competitive  testing.  After  source 
selection  in  October  1980,  the  winning  contractor  was  to  conduct  a 
maturity  effort  on  its  weapon  to  correct  problems  revealed  in  the  com¬ 
petitive  test.  The  Army  then  planned  to  run  a  check  test  in  May  and 
June  1981  to  verify  that  the  problems  had  been  corrected.  Prototypes 
from  both  contractors  were  unexpectedly  immature,  which  caused  the 
30-day  demonstration  to  be  canceled  and  delayed  the  start  of  the  com¬ 
petitive  testing.  The  test  was  completed  about  2  months  late  because  the 
Army  needed  more  time  to  conduct  the  test  and  to  compensate  for 
delays  during  testing.  Under  the  acquisition  strategy,  this  was  the  first 
opportunity  the  Army  had  to  test  the  prototypes,  and  it  appeared  to  be 
surpnsed  at  the  weapons’  immaturity. 

The  Ford  Aerospace  prototypes,  at  the  time  Ford  won  the  competition, 
had  numerous  deficiencies  which  needed  to  be  corrected  before  the 
scheduled  check  test.  The  check  test  did  not  demonstrate  that  all  defi¬ 
ciencies  had  been  overcome.  However,  Army  and  Office  of  the  Secretary 
of  Defense  officials  supported  exercising  the  first  production  option  on 
the  basis  that  most  performance  requirements  had  been  met  and  suffi¬ 
cient  development  progress  had  been  made. 

By  this  time,  the  durability  and  maintainability  test  had  evolved  into  a 
7-month  test  that  would  also  cover  reliability  and  availability.  Though 
this  test  was  originally  scheduled  to  begin  in  February  1982,  the  Army 
delayed  the  start  until  May  1982  to  give  the  contractor  more  time  to 
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upgrade  its  prototype  for  the  test.  Nonetheless,  after  three  attempts  to 
demonstrate  the  prototype’s  readiness  for  production,  the  Army  test 
and  evaluation  agencies  concluded  the  prototype  was  unsuitable  for 
testing  and  discontinued  the  test,  dod  officials  stated  at  the  time  that  the 
test  had  been  terminated  because  certain  prototype  subsystems  were 
close  to  wearing  out  due  to  constant  testing  and  because  the  prototype 
lacked  a  number  of  features  which  would  appear  in  the  production 
model.  However,  reliability  problems  continued  throughout  the  pro¬ 
gram’s  life  when  new  prototypes  and  initial  production  units  were 
tested. 

Two  additional  tests  were  added  to  the  program  before  the  scheduled 
exercise  of  the  third  production  option.  The  engineering  prototype  unit 
test  ran  from  May  through  August  1983  to  assess  the  reliability  of  the 
ammunition  feed  system,  which  had  performed  poorly  during  the  check 
test.  In  addition,  a  design  verification  test  was  conducted  during  1984  to 
complete  production  unit  acceptance  test  procedures  and  to  assess  relia¬ 
bility  and  performance  in  support  of  exercising  the  third  production 
option. 

A  series  of  events  occurred  in  early  1984  which  combined  to  stretch  the 
program  out  further.  Ford  Aerospace  delivered  its  first  production  unit 
in  March  1984,  about  6  months  later  than  called  for  in  the  contract.  The 
contractor  had  encountered  problems  in  making  the  transition  from  pro¬ 
totypes  to  production  units,  including  test  failures,  design  changes, 
rework,  and  assembly  difficulties. 

The  Under  Secretary  of  Defense  for  Research  and  Engineering  then 
I  directed  the  Army  not  to  exercise  the  third  production  option  until  solid 

•  test  results  were  available.  This  decision  was  based  on  the  delay  of  sev¬ 

eral  events  which  were  to  have  occurred  before  the  option  was  exer¬ 
cised,  including  delivery  of  33  production  units,  initisd  production 
testing,  maturity  testing,  and  qualification  testing.  All  these  events  had 
either  not  started  or  their  completion  was  behind  schedule.  The  Under 
Secretary  noted  the  limited  testing  conducted  since  the  initial  production 
decision  had  been  severely  constrained  by  differences  in  the  configura¬ 
tion  of  the  articles  tested,  limited  duration  of  the  tests,  and  unrealistic 
test  conditions.  In  addition,  the  tests  conducted  disclosed  several  tech¬ 
nical  problems.  Consequently,  the  Under  Secretary  directed  the  Army  to 
conduct  a  realistic  period  of  operational  testing  before  exercising  the 
third  production  option. 
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Insufficient  Information  for 
Dedsionmakers 


I 


I 


The  Army  conducted  this  testing  in  July  1984,  referred  to  as  the  “lim¬ 
ited  test.”  Test  results  were  inconclusive  because  of  the  limited  realism 
of  the  test  conditions  and  the  lack  of  statistically  significant  numbers. 
Then  the  Secretary  of  Defense  decided  to  defer  exercising  option  III  and 
to  delay  initiating  phase  III  of  the  program  until  after  additional  opera¬ 
tional  tests,  which  were  completed  in  June  1986.  This  testing  was  to  be 
the  first  of  two  follow-on  evaluations.  The  second  series  would  have 
been  held  in  1987  had  the  program  continued. 


Two  features  of  the  Sergeant  York’s  acquisition  strategy — the  hands-off 
management  by  the  Army  during  phase  I  and  limited  testing — at  times 
made  it  difficult  to  justify  funding  requests  for  the  program  and,  as  a 
result,  contributed  to  program  delays. 

In  May  1978,  the  Sergeant  York  project  manager  cited  several  potential 
risks  inherent  in  the  program,  including  a  concern  that  information  on 
the  progress  of  the  development  program  might  be  inadequate  to  justify 
funding  needs  to  the  Office  of  the  Secretary  of  Defense,  the  Office  of 
Management  and  Budget,  and  the  Congress.  The  minimum  government 
involvement  and  the  limited  testing,  particularly  during  the  competition, 
precluded  in-depth  knowledge  and  release  of  technical  performance  data 
on  the  system  under  development.  These  constraints  inhibited  essential 
information  on  system  performance  and  program  progress  from 
reaching  those  who  were  to  make  funding  and  programming  decisions. 
This  lack  of  data  contributed  to  significant  reductions  in  the  Army’s 
1981  planned  budget  and,  in  turn,  to  a  major  program  restructuring.  The 
Sergeant  York’s  funding  was  cut  back  to  the  extent  of  deleting  the  fiscal 
year  1981  buy,  the  first  year’s  procurement.  The  restructured  program 
delayed  the  production  decision  by  18  months  and  added  17  months  to 
the  initial  operational  capability  date. 

Deleting  the  fiscal  year  1986  buy  and  deferring  exercising  the  third  pro¬ 
duction  option  were  due  to  the  Sergeant  York’s  performance  in  testing 
and  to  the  uncertainty  about  the  system  stemming  from  the  lack  of  suf¬ 
ficient  operational  testing  on  production  units.  Defense  officials  felt  that 
more  operational  testing  was  needed  than  had  been  planned  under  the 
acquisition  strategy.  Thus,  although  the  acquisition  strategy  called  for 
little  operational  testing  to  be  done  before  initial  production  such  testing 
proved  to  be  necessary  to  continuing  the  program. 
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Perforlnance  in  Testing  Was 
Unsatisfactory 


Importaiit  Specifications  Not  Met 


Army’s  Assessments  of  Operational 
Test  Results  Were  More  Favorable 
'Than  thb  Secretary  of  Defease’s 


Although  the  Sergeant  York  had  undergone  numerous  tests  since  it 
began  development,  some  important  tests  had  been  delayed  and  the  lim¬ 
ited  opierational  test  added  in  the  summer  of  1984  at  the  direction  of  the 
Office  of  the  Secretary  of  Defense  proved  to  be  inconclusive. 

The  most  definitive  tests  of  Sergeant  York  were  those  completed  in  mid- 
1985.  The  initial  production  test,  conducted  from  December  1984  to  May 
1986  at  three  test  sites,  measured  the  Sergeant  York’s  performance 
against  contract  specifications.  The  operational  test,  conducted  from 
April  to  June  1985  at  two  test  sites,  determined  its  capability  in  a  simu¬ 
lated  combat  environment. 


The  Army  Materiel  Systems  Analysis  Activity,  in  evaluating  the  initial 
production  tests,  found  that  the  Sergeant  York  either  met  or  exceeded 
141  of  the  163  contract  specifications  tested.  Of  the  22  that  were  not 
met,  7  were  considered  by  the  agency  to  be  particularly  significant. 
These  concerned  the  reliability  of  the  gun’s  power  and  actuation  sub¬ 
system,  its  survivability  against  certain  countermeasures,  a  safety 
hazard  involving  the  turret’s  rotation,  a  problem  with  the  proximity 
fuze,  and  the  gun’s  performance  against  certain  threats. 

The  power  and  actuation  mission  reliability  specification  called  for  a 
mean  time  between  failures  of  260  hours.  "The  Sergeant  York  achieved 
only  36  hours  between  failures  during  the  test.  The  Army  agency  esti¬ 
mated  that  the  entire  gun  system  mission  reliability  for  the  test  was  21 
hours  of  operation  between  failures  as  compared  with  the  specified  37 
hours. 

Two  shortfalls  involved  the  inability  of  the  Sergeant  York  to  meet  speci¬ 
fications  for  correctly  identifying  certain  incoming  objects.  Details  of 
this  performance  deficiency  are  classified.  The  turret  problem  concerned 
the  lack  of  an  audible  cue  to  warn  the  driver  of  the  turret’s  rotation — a 
safety  hazard  when  the  driver’s  head  was  exposed.  Other  specification 
shortfalls  concerned  problems  with  the  performance  of  the  acquisition 
and  tracking  radar. 


Based  on  their  respective  eissessments  of  the  operational  test  results,  the 
Office  of  the  Secretary  of  Defense  and  the  Army  reached  different  con¬ 
clusions.  The  former  concluded  that  the  Sergeant  York  had  been  ineffec¬ 
tive  in  performing  the  air  defense  mission  and  represented  only  a 
marginal  improvement  over  the  Vulcan  gun.  The  Army  concluded  that 
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the  Sergeant  York  was  significantly  better  than  the  Vulcan  in  protecting 
friendly  forces  but  was  ineffective  against  hovering  helicopters. 

In  planning  the  operational  test,  the  Army’s  Training  and  Doctrine  Com¬ 
mand  developed  criteria  for  measuring  the  Sergeant  York’s  effectiveness 
in  performing  its  mission.  The  determining  factor  was  to  be  the  friendly 
force’s  survival  while  Sergeant  York  was  participating  in  its  defense 
compared  with  the  survival  when  the  current  Vulcan  air  defense  gun 
was  a  participant.  The  Sergeant  York  was  to  be  judged  effective  if 
friendly  losses  to  air  attack  were  27  to  36  percent  lower  than  losses  sus¬ 
tained  while  the  Vulcan  was  protecting  the  force. 

The  test  was  divided  into  a  series  of  trials,  each  lasting  30  minutes. 

There  were  a  total  of  61  valid  trials.  Army  test  evaluators  were  con¬ 
cerned  that  the  small  number  of  trials  might  not  be  statistically  signifi¬ 
cant.  In  addition,  evaluators  of  the  Army’s  Combat  Developments 
Experimentation  Center,  which  made  the  first  of  three  Army  evalua¬ 
tions,  questioned  the  validity  of  the  criteria  because  of  the  large  number 
of  variable  conditions  both  within  and  among  the  trials.  The  center  sug¬ 
gested  that  an  analysis  of  the  results  based  on  the  number  of  opportuni¬ 
ties  the  two  guns  had  to  engage  aircraft  during  the  trials  would  provide 
a  statistically  significant  and  more  homogeneous  sample  than  an  anal¬ 
ysis  based  on  the  trials.  According  to  evaluators  from  the  center,  there 
were  over  1,200  such  opportunities  when  the  Sergeant  York  was  oper¬ 
ating.  The  center  analyzed  the  data  using  both  trials  and  engagement 
opportunities  but  gave  more  weight  to  the  latter.  Its  evaluators  con¬ 
cluded  that  the  Sergeant  York  had  been  ineffective  in  performing  its 
mission. 

The  center’s  report  was  provided  to  the  Army  Operational  Test  and 
Evaluation  Agency  for  consideration  in  its  independent  assessment  of 
the  operational  test  The  agency’s  evaluators  made  their  assessment  in 
accordance  with  the  criteria  developed  by  the  Training  and  Doctrine 
Command  to  evaluate  the  Sergeant  York’s  performance.  However,  the 
agency  eliminated  some  of  the  trials  from  its  analysis  because  the  air¬ 
craft  in  those  particular  trials  were  judged  not  to  pose  a  threat  to 
friendly  forces.  As  a  result,  the  agency’s  analysis  was  based  on  a  smaller 
sample  size  than  the  one  the  Combat  Developments  Experimentation 
Center  questioned.  The  Operational  Test  and  Evaluation  Agency  con¬ 
cluded  that  the  Sergeant  York  met  the  mission  performance  criteria 
although  it  recognized  the  limitations  of  the  criteria. 
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The  Operational  Test  and  Evaluation  Agency  did  find  that  the  Sergeant 
York  did  not  meet  prescribed  operational  requirements  for  availability 
and  maintainability.  The  weapon  was  to  be  available  for  operations  90 
percent  of  the  time.  In  the  test,  its  operational  availability  was  not 
better  than  64  percent.  This  did  not  include  downtime  while  waiting  for 
logistics  support  and  for  administrative  purposes.  The  agency  estimated 
that  had  these  been  considered,  the  gun’s  availability  would  have  been 
only  33  percent.  The  Sergeant  York’s  mean  time  to  repair  was  required 
to  0.5  hours  or  less  at  the  organizational  level  and  1  hour  or  less  at 
the  direct  support  level.  As  tested,  the  mean  time  to  repair  the  system  at 
all  levels  was  2.01  hours. 

The  third  Army  evaluation,  also  based  on  the  trial  results,  was  com¬ 
pleted  in  the  Office  of  the  l^retary  of  the  Army  on  August  22, 1985. 
This  evaluation  represented  the  Army’s  official  position.  It  found  the 
Sergeant  York’s  performance  to  be  significantly  better  than  the 
Vulcan’s,  although  some  system  shortcomings  still  required  correcting.  It 
did  not  recognize  the  limitations  of  the  criteria  as  had  the  previous 
evaluations. 

In  commenting  on  a  draft  of  this  report,  dod  stated  that  the  Army’s  eval¬ 
uation  of  the  test  results  had  not  proceeded  through  three  distinct  levels 
but  had  been  a  total  product  based  on  established  procedures.  It  stated 
that  the  Army’s  Combat  Developments  Experimentation  Center  report 
was  not  meant  to  be  a  final  report  because  a  detailed  analysis  of  all  the 
trials  was  not  yet  available,  dod  further  stated  that  while  it  nught 
appear  that  the  Army’s  final  assessment,  conducted  by  an  independent 
Army  assessment  team,  was  more  favorable  than  the  initial  evaluation, 
additional  and  more  valid  test  results  and  technical  evaluations  were 
available  at  the  time  of  the  final  assessment. 


Office  of  the  Secret^  of  Defense  Evaluators  of  the  newly  created  Office  of  Operational  Test  and  Evalua- 
Found  jThat  Operational  tion  in  the  Office  of  the  Secretary  of  Defense,  reviewing  the  same  data 

Requirements  Had  Not  Been  Met  Army  had  evaluated,  found  that  although  the  Sergeant  York  rep¬ 

resented  an  improvement  over  the  current  Vulcan  air  defense  gun,  it  did 
not  adequately  protect  friendly  forces  during  simulated  combat.  They 
1  also  found  that  the  Sergeant  York  was  not  operationally  suitable 

because  of  its  low  availability  stemming  from  its  being  frequently  down 
I  for  maintenance. 

According  to  Office  of  the  Secretary  of  Defense  officials,  the  Army’s 
operational  test  results  demonstrated  that  the  Sergeant  York  could  not 
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effectively  meet  the  Soviet  helicopter  threat.  Helicopters,  when  they 
stood  off  beyond  the  gun’s  range,  effectively  delivered  firepower  against 
the  friendly  forces.  Overall,  the  gun  brought  down  an  unacceptably 
small  number  of  helicopters  and  the  forces  it  was  protecting  suffered 
large  losses. 


Objectives,  Scope,  and 
Methodology 


Our  objectives  were  to  (1)  examine  the  acquisition  strategy  established 
for  the  Sergeant  York  and  determine  to  what  extent  it  had  been  helpful 
in  enabling  the  Army  to  achieve  its  acquisition  objectives,  (2)  determine 
the  degree  of  protection  that  the  Sergeant  York  contract’s  warranty  pro¬ 
visions  provided  the  government,  particularly  with  respect  to  containing 
program  cost,  and  (3)  determine  to  what  extent  the  Sergeant  York  was 
meeting  its  performance  requirements. 


To  accomplish  these  objectives  we  examined  pertinent  documentation 
from  the  Department  of  Defense,  the  Army,  and  the  contractor.  The  key 
documentation  examined  included 


•  Army  testimony  on  the  acquisition  strategy  and  program  progress; 

•  the  Army’s  required  operational  capability; 

•  the  phase  II  contract  for  system  maturity  and  initial  production,  with 
particular  attention  to  its  development  specification  and  the  warranty 
provisions;  and 

•  Army  and  contractor  test  reports,  as  well  as  Army  and  Department  of 
Defense  evaluations  of  those  tests. 

We  performed  our  work  primarily  at  the  Army’s  Sergeant  York  project 
office,  Dover,  New  Jersey.  We  also  held  discussions  with  representatives 
of  the  U.S.  Army  Operational  Test  and  Evaluation  Agency,  Falls  Church, 
Virginia;  the  U.S.  Army  Materiel  Systems  Analysis  Activity,  Aberdeen 
Proving  Ground,  Maryland;  the  U.S.  Army  Air  Defense  Center,  Fort 
Bliss,  Texas;  the  U.S.  Army  Combat  Developments  Experimentation 
Center,  Fort  Ord,  California;  the  U.S.  Army  Office  of  Missile  Electronic 
Warfare  and  the  U.S.  Army  Training  and  Doctrine  Command  Systems 
Analysis  Activity,  White  Sands  Missile  Range,  New  Mexico;  the  Office  of 
the  Secretary  of  Defense,  the  Pentagon;  and  Ford  Aerospace  and  Com¬ 
munications  Corporation,  Newport  Beach,  California.  We  also  observed 
Sergeant  York  testing  at  the  Army’s  North  McGregor  Range,  Fort  Bliss, 
Texas;  Fort  Hunter  Liggett,  California;  and  Red  Rio  Area,  White  Sands 
Missile  Range,  New  Mexico. 
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We  began  our  review  in  June  1984  and  completed  it  in  August  1986.  Our 
review  was  performed  in  accordance  with  generally  accepted  govern¬ 
ment  auditing  standards. 


I 
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NCSEARCH  AND 
fNC.INtERING 


THE  UNDER  SECRETARY  OF  DEFENSE 


WASHINGTON  DC  ^OJOl 

3  m  1986 


Mr.  Frank  C.  Conahan 
Director 

National  Security  and 

International  Affairs  Division 
U.S.  General  Accounting  Office 
Washington,  D. C,  20548 


Dear  Mr.  Conahan: 

This  IS  the  Department  of  Defense  (DoD)  response  to  the 
General  Accounting  Office  (GAO)  draft  report,  ’’SERGEANT  YORK: 
Concerns  About  the  Army’s  Accelerated  Acquisition  Strategy,” 
dated  January  8,  1986  (GAO  Code  393004/OSD  Case  6915). 

The  DoD  generally  concurs  with  the  draft  report.  Detailed 
comments  on  the  report  findings  and  recommendations  are  provided 
in  the  enclosure.  The  Department  appreciates  the  opportunity  to 
comment  on  the  draft. 


Sincerely, 
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GAO  DRAFT  REPORT  -  DATED  JANUARY  8*  1986 
(GAO  CODE  393004)  -  OSD  CASE  6915 

"SERGEANT  YORK:  CONCERNS  ABOUT  THE  ARMY'S  ACCELERATED 
ACQUISITION  STRATEGY" 

DEPARTMENT  OF  DEFENSE  COMMENTS 


FINDINGS 


•  FINDING  At  Features  Of  TOa  Acquisition  Strategy,  The  GAO 

noted  that  the  SfiRCSEANT  VOfeK  acquisition  strategy  was  unique 
in  that  it  attempted  to  accelerate  fielding,  and  at 
substantial  cost  savings  to  the  Government.  The  GAO  also 
noted  that  the  features  of  the  acquisition  strategy 
included : 

parallel,  competitive  development; 

a  "skunk  works"  approach  to  contracting  in  which 
contractors  were  given  free  rein  to  design  the  system 
to  meet  the  Army's  requirements; 

--  the  "hands-off"  approach  in  that  the  Government's 
management  role  was  minimal; 

combining  development  and  operational  testing  prior  to 
production;  and 

--  combining  the  systems'  follow-on  development  with 
initial  production. 

The  GAO  found  that  these  factors  were  intended  to 
significantly  shorten  the  development  phases  preceding 
initial  production.  The  GAO  also  found  that  Army  recognized 
that  this  approach  had  technical  and  cost  risks.  The  GAO 
additionally  found  that  to  minimize  these  risks,  the  Army 
specified  that  contractors  use  already  proven  subsystems  in 
weapon  design  and  negotiated  (1)  a  fixed-price  development 
contract  with  fixed-price  production  options  and 
(2)  warranty  provisions  which  protected  the  Government 
against  cost  increases  resulting  from  corrections  of  design 
and  other  deficiencies.  The  GAO  noted  that  the  Secretary’s 
August  27,  1985  decision  to  cancel  the  $4.04  billion 
procurement  of  614  gun  systems  was  based  on  the  weapon's 
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Nowonpp  1-2,  6-7,  and 
14-18 


Nowonpp  3 and  15-16 


inability  to  effectively  protect  friendly  forces,  and 
because  he  did  not  believe  the  marginal  improvement  over 
existing  weapon  systems  was  worth  the  large  investment  it 
would  take  to  complete  the  program.  The  GAO  concluded  that 
the  Army  should  continue  its  efforts  to  reduce  the  time  it 
takes  to  develop  and  field  new  weapons,  but  there  are 
lessons  to  be  learned  from  the  events  that  followed  the 
application  of  the  accelerated  strategy.  (p.  1,  p.  3,  pp. 
9-11,  and  pp.  1-3,  p.  7,  Appendix  I,  GAO  Draft  Report) 

DoP  POSITION >  Concur. 

FINDING^*  Conpetitive  Prototyi^ing.  The  GAO  noted  that 
the  first  phase  of  the  acquisition  strategy  was  competition 
between  two  contractors,  with  each  delivering  complete 
prototypes  to  the  Army  for  testing.  The  GAO  found  that  the 
advantages  of  this  approach  were:  (1)  having  more  than  one 
candidate  for  the  mission,  (2)  having  actual  hardware  on 
which  to  base  a  comparison,  and  (3)  stimulating  individual 
contractor  creativity  in  devising  a  weapon  to  fulfill  the 
mission.  The  GAO  found  that  the  Army  decided  to  select  a 
contractor  for  the  phase  II  full  scale  development  and 
initial  production  contract  after  the  competitive 
development  and  operational  tests  were  completed  and 
evaluated.  The  GAO  also  found  that,  at  the  time,  both 
prototypes  displayed  numerous  shortcomings  and  still 
required  considerable  development.  The  GAO  concluded  that, 
as  a  result,  (1)  some  of  the  advantage  of  the  competition 
may  have  been  lost,  and  (2)  it  may  have  been  prudent  to  wait 
for  the  further  development  of  both  prototypes  before  making 
a  selection.  (p.  4,  and  pp.  3-4,  Appendix  I,  GAO  Draft 
Report ) 

DoD  POSITIOWi  Concur. 

FINDING  Ci  Miniau»  Governaent  Involvenent  And  Limited 
Concurrent  Testing  Provided  Insuf f iclent  Jtnfornatlon  For 

Decisionmakera J  T^e  GA6"~found~ that  the  ^Tiands  off"  strategy 
embodied  (1)  having  thinly  staffed  Government  and  contractor 
project  offices  shielded  from  interference  by  other 
concerned  Government  officials,  (2)  reducing  the  reporting 
requirements,  (3)  limiting  access  to  contractors  and 
contractor  information,  and  (4)  giving  the  contractors 
flexibility  to  design  and  develop  a  weapon  that  would 
satisfy  Army  requirements.  Tlie  GAO  found  that  the  Army 
chose  to  (1)  perform  developmental  testing  concurrently  with 
production  and  (2)  conduct  less  testing  before  the 
production  decision  than  is  usually  done  under  a  more 
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conventional  acquisition  strategy.  The  GAO  noted  that 
developmental  tests,  deferred  until  after  production  began, 
included  operations  in  a  cold  climate,  durability  in 
stressful  battlefield  conditions,  system  transportability, 
and  electromagnetic  tests.  The  GAO  found  that,  although 
operational  tests  were  scheduled  before  production,  they 
were  constrained  by  the  limited  realism  of  the  test 
environment.  GAO  concluded  that  the  "hands-off”  management 
strategy  by  the  Army  during  phase  I  and  the  limited  testing 
did  not  provide  essential  performance  information,  making  it 
difficult  to  justify  funding  requests  for  the  program  and, 
as  a  result,  contributed  to  program  delays.  The  GAO  further 
concluded  that  the  minimum  Government  involvement, 
particularly  during  the  competition,  precluded  in-depth 
knowledge  and  release  of  technical  information,  inhibiting 
essential  data  on  system  performance  from  reaching  those  who 
were  to  make  funding  and  programming  decisions.  The  GAO 
additionally  concluded  that,  because  the  strategy  limited 
the  opportunities  for  testing,  it  did  not  give 
decisionmakers  as  mucli  information  at  given  points  in  time 
as  they  normally  receive  under  more  conventional  strategies. 
The  GAO  finally  concluded  that  confining  surveillance  over 
the  system’s  development  early  in  the  program  to  just  a  few 
Army  officials  provided  little  opportunity  to  hold  the 
program  up  to  wider  and  more  critical  scrutiny  in  the  Army's 
higher  echelons ,  the  OSD  and  the  Congress .  (p .  2 ,  pp ,  4-8 , 
p.  10,  and  pp.  16-21,  Appendix  I,  GAO  Draft  Report) 

DoD  POSITION 5  Concur . 

FINDING  D:  Protection  Offered  By  Warranty  Provisions.  The 

GAO  found  that  the  contract  to  acquire  the  VORK 

provided  protection  from  cost  increases  caused  by  defects  in 
the  design,  component  integration,  materials  or  workmanship. 
This  protection,  according  to  the  GAO,  went  well  beyond  that 
generally  obtained  for  other  weapon  systems  where  the 
contractor's  liability  does  not  cover  design  defects. 
Specifically,  the  GAO  noted,  the  warranty  provisions  made 
the  contractor  responsible  for  correcting  the  defects 
without  increases  to  the  contract's  target  cost,  target 
ceiling  or  ceiling  price,  and  without  extensions  of  delivery 
time.  The  GAO  also  noted  that  under  the  warranty,  the 
performance  of  the  system  was  to  be  measured  against 
detailed  requirements  in  the  contract's  development 
specifications.  The  GAO  found  that  the  contractor's 
warranty  did  not  extend  to  performance  of  the  system  in  an 
operational  environment ;  instead ,  these  requirements 
(mission  performance  and  survivability)  were  to  be 
demonstrated  under  test  conditions  which  represented  a 
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realistic  battlefield  environment.  The  GAO  also  found  that 
the  contractor  would  not  have  been  obligated  under  the 
warranty  to  correct  deficiencies  disclosed  in  this  test. 

The  GAO  concluded,  however,  that  it  seems  probable  the 
benefits  derived  from  having  a  warranty  requiring  a  system 
to  perform  to  specification  in  a  nonoperational  environment 
will,  to  a  certain  extent,  carry  over  to  its  performance  in 
an  operational  environment.  (pp.  8-9,  p.  11,  and  pp.  7-8, 
Appendix  I,  GAO  Draft  Report) 


Now  on  pp  4  and  19  20 


POP  POSITIOMi  Concur . 

FINDING  E;  Cost  Control  Was  Maintained.  The  GAO  found  that 
the  fixed-price  contract  was viewed  as  complementing  the 
"hands-off"  and  tradeable  requirements  features  by  eliciting 
the  contractors'  best  efforts  for  a  given  price,  and  that 
the  contract  included  fixed  price  production  options  made  by 
each  contractor  during  the  competition.  The  GAO  found  that, 
until  the  program's  termination  in  August  1985,  costs  to  the 
Government  had  remained  relatively  stable.  The  GAO 
concluded  that  this  was  a  significant  achievement, 
considering  the  pattern  of  cost  growth  exhibited  by  many 
weapons  during  their  development  and  early  production.  The 
GAO  also  found  that  while  the  competitive  development  of 
integrated  prototypes  and  use  of  proven  components 
contributed  to  this  favorable  cost  experience,  the  fact  that 
significant  technical  problems  still  occurred  suggests  the 
use  of  the  fixed-price  development  contract  and  production 
contract  options  were  effective  cost  control  features  of  the 
acquisition  strategy.  The  GAO  found,  however,  the  fixed- 
price  options  did  have  a  drawback  in  that  they  put  pressure 
on  decisionmakers  to  proceed  with  production  on  schedule  to 
take  advantage  of  favorable  pricing,  despite  the  system's 
technical  difficulties  and  unknowns.  The  GAO  concluded  that 
a  major  challenge  to  the  Army's  ability  to  maintain  the 
program's  cost  control  would  have  come  with  the  negotiation 
of  the  first  full  production  contract  scheduled  to  be 
awarded  in  January  1986.  According  to  GAO,  that  contract 
was  to  be  negotiated  with  the  benefit  of  actual  production 
experience,  but  without  the  price  protection  the  fixed  price 
options  had  provided.  The  GAO  further  concluded  that 
cancellation  of  the  program  leaves  unanswered  the  question 
of  whether  cost  control  would  have  continued.  (pp.  5-6,  and 
pp,  9-11,  Appendix  I,  GAO  Draft  Report) 

POP  POSITION:  Concur. 


FINDING  Fz  Accelerated  Deployent  Not  Achieved.  The  GAO 

found  tViat  the  acquisition  strategy  did  not  achieve  early 
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Now  on  pp  4-6  and  21  25 


deployment  of  the  SERGEANT  YORK.  In  addition,  the  GAO  found 
that  the  gains  in  time  the  Army  anticipated  from  combining 
development  testing  and  initial  production  were  never 
realized.  The  GAO  noted,  for  example,  that  the  system  would 
not  have  reached  operational  capability  until  10  years  after 
program  initiation,  compared  with  the  6-^  years  originally 
planned.  The  GAO  further  found  that  the  major  causes  of 
this  delay  appeared  to  be  (1)  deletion  of  fiscal  year  1981 
production  due  to  funding  cuts,  which  delayed  the  first 
year's  production  by  about  1-^  years  and  (2)  production 
start-up  problems,  which  delayed  deliveries.  The  GAO  found, 
however,  that  the  10  year  interval  is  still  somewhat  leas 
than  was  experienced  by  seven  other  weapon  systems  the  GAO 
used  for  comparison  purposes.  These  other  systems, 
according  to  the  GAO,  acquired  under  more  conventional 
strategies,  averaged  12  years  from  program  initiation  to 
operational  capability.  In  another  comparison,  the  GAO 
found  that  the  Multiple  Launch  Rocket  System,  using  an 
acquisition  strategy  similar  to  the  SERGEANT  YORK'S,  was 
fielded  is  just  over  six  years,  or  only  four  months  later 
than  planned.  The  GAO  concluded  that  several  factors  appear 
to  explain  why,  under  a  strategy  similar  to  the  SERGEANT 
YORK,  a  system  was  fielded  essentially  within  six  yeras. 
First,  the  GAO  noted,  that  the  rocket  system  is  made  up  of 
previously  applied  technology  and  is  relatively 
uncomplicated.  Second,  time  was  set  aside  for  final 
operational  testing  prior  to  the  full-scale  production 
decision,  whereas,  the  SERGEANT  YORK  strategy  did  not 
provide  for  such  testing  until  after  full-scale  production 
was  approved.  Third,  although  the  rocket  system  exhibited 
reliability  and  accuracy  problems  even  when  full-scale 
production  began,  the  program  was  not  delayed  to  correct  the 
problems,  as  was  done  with  the  SERGEANT  YORK.  The  GAO 
concluded  that  the  desired  acceleration  in  fielding  the 
SERGEANT  YORK  was  not  achieved  due  to  both  weaknesses  in  the 
acquisition  strategy  and  the  technical  difficulties  of  the 
weapon  itself,  although  it  is  difficult  to  sort  out  the 
relative  contribution  of  each.  The  GAO  also  concluded, 
however,  that  the  technical  problems  of  the  system  were  not 
due  to  the  acquisition  strategy,  and  that  technical 
difficulties  can  be  encountered  in  developing  any  complex 
system,  irrespective  of  the  acquisition  strategy  used.  The 
GAO  finally  concluded  that  while  the  SERGEANT  YORK 
acquisition  strategy  contained  some  positive  elements,  on 
the  whole,  it  was  detrimental  to  the  program,  (pp.  6-7, 
p.  9,  and  pp.  12-16,  Appendix  I,  GAO  Draft  Report) 

DoD  POSITION:  Concur.  A  third  cause  of  the  delay  was  the 
requirement  to  do  additional  operational  testing  before 
exercising  the  FY  1984  (third)  production  option  of  the 
Phase  II  contract. 
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•  FIMDIMG  Gt  Perforaance  In  Testing  Was  Unsat in factory  And 
laporiant  Specifications  Hot  Met.  The  GAO found  that  the 
system  test  units  were  less  technically  raature  than 
anticipated  during  the  program,  giving  rise  to  testing 
delays  and  the  need  for  additional  testing.  The  GAO 
concluded  that  the  system's  major  subsystems,  as  well  as 
their  application  to  a  weapon  for  which  they  were  not 
originally  designed,  represented  a  greater  technical 
undertaking  than  anticipated.  The  GAO  also  found  that  the 
contractor  encountered  problems  in  making  the  transition 
from  prototypes  to  production  units,  including  test 
failures,  design  changes,  rework,  and  assembly  difficulties. 
The  GAO  found  that,  as  a  result,  the  OSD  directed  the  Army 
not  to  exercise  the  third  production  option  until  solid  test 
results  were  available,  including  realistic  operational 
testing ,  The  GAO  noted  that  additional  limited  tests  were 
conducted ,  but  the  results  were  inconclusive  because  of 
limited  realism  of  the  test  and  lack  of  statistically 
significant  numbers.  According  to  the  GAO,  further  tests 
would  have  been  held  in  1987,  had  the  program  continued. 

The  GAO  found  that  the  most  definitive  tests  were  completed 
in  mid~1985,  and  these  initial  production  and  follow-on 
operational  tests  were  the  first  attempts  to  assess  the 
system  in  a  realistic  environment.  The  GAO  noted  that  the 
analysis  performed  by  the  DoD  Director,  Operational  Test  and 
Evaluation,  while  acknowledging  improvement  over  the  current 
Vulcan  air  defense  gun,  concluded  that  the  SERGEANT  YORK  was 
ineffective  in  protecting  friendly  forces  and  unsuitable  for 
combat  operations  because  of  its  low  operational 
availability.  The  Army,  however,  while  noting  that  some 
shortcomings  remained,  ultimately  concluded  that  the  results 
of  initial  production  and  operational  tests  were  favorable, 
including  the  determining  test  factor — friendly  force's 
survival.  The  GAO  reported  that  the  Army  conducted  three 
levels  of  review  of  the  test  results  and  concluded  that,  as 
the  evaluations  progressed  through  the  various  levels,  the 
results  became  more  favorable.  The  GAO  noted,  for  example, 
that  the  third  Army  evaluation,  based  on  the  trial  results, 
was  completed  in  the  office  of  the  Secretary  of  the  Army  on 
August  22,  1985.  According  to  the  GAO,  it  was  advised  that 
this  represented  the  Army's  official  position  and  that  the 
Army  found  the  SERGEANT  YORK'S  performance  to  be  very 
favorable.  The  GAO  concluded  that  the  limited  Army 
surveillance,  coupled  with  confidence  in  the  use  of  proven 
components  fueled  optimism  about  the  system's  eventual 
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performance  and  enabled  the  initial  production  decision  to 
be  made  on  the  basis  of  limited  testing,  although  subsequent 
test  results  showed  these  views  to  be  overly  optimistic. 

The  GAO  also  concluded  that  the  wisdom  of  the  Secretary's 
decision  to  cancel  the  program  was  borne  out  by  the  tests 
Now  on  pp  5, 7  and  25-31  results.  (pp.  7-8,  p.  10,  and  pp.  16-24,  Appendix  I,  GAO 

Draft  Report) 

Pod  POSITION;  Partially  Concur.  The  DoD  concurs  with  the 
GAO  conclusion  that  the  Secretary  of  Defense's  decision  to 
terminate  the  SERGEANT  YORK  was  borne  out  by  the  test 
results.  The  GAO  explanation  of  how  the  test  was  evaluated 
by  the  Army  needs  clarification,  however.  The  Army's 
I  evaluation  of  SERGEANT  YORK  did  not  state  that  the  results 

I  were  favorable.  The  Army  stated  that  SERGEANT  YORK  was 

significantly  better  than  VULCAN  in  terms  of  blue  force 
survival,  but  proved  to  be  ineffective  against  a  current 
threat  related  to  hover  mg  helicopters  standing  off  at 
ranges  greater  than  those  for  which  the  system  was  designed. 

I  Further,  the  Army's  evaluation  of  the  test  results  did  not 

'  proceed  through  three  distinct  levels,  but  was  evaluated  in 

total  through  established  Army  review  procedures.  Due  to 
I  the  enormous  amount  of  data  collected  during  the  operational 

I  test,  the  Army's  independent  operational  tester,  Operational 

Test  and  Evaluation  Agency  (OTEA) ,  formed  a  Data  Analysis 
I  Group  to  collect,  reduce,  analyze  and  evaluate  the  test 

,  I  results.  The  Army's  Combat  Development  Experimentation 

Center  (CDEC)  report  was  an  early  quick  look  assessment  of 
the  results.  It  was  not  meant  to  be  a  final  report  because 
a  detailed  analysis  of  all  the  trails  was  not  yet  available. 
After  the  completion  of  the  operational  test  in  June  1985, 
the  Army  formed  an  independent  assessment  team  that  was 
tasked  to  look  at  the  operational  test  resul ts/evalua t ion 
^  and  make  its  own  assessment  of  the  SERGEANT  YORK  in  order 

•  for  It  to  accomplish  its  air  defense  mission.  While  it  may 

appear  that  the  final  assessment  was  more  favorable  than  the 
initial  evaluation,  it  must  be  kept  in  mind  that  additional 
and  more  valid  test  results  and  technical  evaluations  were 
available  by  the  time  of  the  Army  briefings  to  the  Secretary 
of  the  Army  and  Secretary  of  Defense  on  August  22,  1985, 


RECOMMENDATIONS 

•  RECOMMENDATION  1;  The  GAO  recommended  that  when  the  Army 

consider  such  strategies  for  future  programs,  the  Secretary 
of  Defense  require  the  Army  to  make  an  initial  detailed 
assessment  of  the  technical  risks  involved  in  individual 
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subsystems ,  as  well  as  in  the  integration  of  those 
subsystems  into  one  system,  with  an  explicit  focus  on 
whether  the  technology  being  attempted  is  compatible  with  an 
accelerated  acquisition  strategy.  (According  to  the  GAO, 
built  into  the  strategy  should  be  provisions  for  adjusting 
schedules  and  other  program  facets,  if  technical 
Now  on p  8  difficulties  occur.)  (p.  11,  GAO  Draft  Report) 

DoD  POSITION;  Concur.  No  additional  guidance  is  required, 
however ,  because  the  essence  of  the  GAO  recommendation  was 
in  effect  at  the  time  decisions  were  made  on  the  SERGEANT 
YORK  program.  In  the  case  of  the  SERGEANT  YORK,  a  conscious 
decision  was  made  by  the  DoD,  with  the  knowledge  of  the 
Congress ,  to  use  a  unique  acquisition  strategy  with 
I  recognized  risks.  In  retrospect,  specific  lessons  have  been 

I  learned,  particularly  with  respect  to  the  "hands  off" 

aspects  of  the  strategy .  These  lessons ,  coupled  with  the 
DoD  Inspector  General’s  Audit,  "Effectiveness  of  the  Defense 
Systems  Acquisition  Review  Council  Process,"  and  the 
procedures  in  applicable  DoD  directives,  should  preclude  a 
similar  situation  from  occurring  in  the  future. 

The  DoD  Instruction  5000.2,  "Major  Systems  Acquisition 
Procedures,"  already  requires,  as  early  as  Milestone  0,  that 
the  maturity  of  the  technology  planned  for  the  selected 
system  be  discussed  with  an  assessment  of  risk  in  the 
*  Justification  For  Major  System  New  Start  (JMSNS).  Also  in 

the  System  Concept  Paper  (SCP)  and  Decision  Coordinating 
Paper  (DCP),  for  Milestones  I  and  Il/lII,  respectively,  the 
technological  risks,  as  well  as  the  general  acquisition 
strategy  for  the  entire  program  and  the  detailed  acquisition 
strategy  for  proceeding  to  the  next  milestone,  are  required 
to  be  discussed.  Further,  the  DoD  Directive  5000.1,  "Major 
Systems  Acquisition,"  requires  that  the  acquisition  strategy 
I  tailor  the  prescribed  steps  in  the  major  system  acquisition 

^  decision-making  process  to  the  strategy.  DoD  Directive 

4245.7,  "Transition  from  Development  to  Production," 
requires  a  formal  program  of  risk  assessment  and  authorizes 
a  recently  published  manual,  which  provides  guidance  on  ways 
to  reduce  risk  to  acceptable  levels. 

•  RECOMMENDATION  2s  The  GAO  recommended  that  when  the  Army 

considers  such  strategies  for  future  programs,  the  Secretary 
of  Defense  require  the  Army  to  assess  the  weapon's 
technological  progress  periodically  to  see  if  it  is  still 
compatible  witli  the  planned  acceleration.  (In  the  view  of 
the  GAO,  1 f  technical  progress  is  no  longer  keeping  pace 
j  with  the  acceleration,  the  strategy  should  be  adjusted  to 

Now  oh p  8  '  bring  it  in  line  with  the  technology.)  (p.  12,  GAO  Draft 

Report ) 


! 
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DoD  TOSITION;  Concur*  Again,  no  additional  guidance  is 
required  because  the  essence  of  the  GAO  recommendation  was 
in  effect  at  the  time  decisions  were  made  on  the  SERGEANT 
YORjK  program.  In  the  case  of  the  SERGEANT  YORK,  a  conscious 
decision  was  made  by  the  DoD,  with  the  knowledge  of  the 
Congress,  to  enter  a  unique  acquisition  strategy  with 
recognized  risks.  In  retrospect,  specific  lessons  have  been 
learned,  particularly  with  respect  to  the  "hands  off"  aspect 
of  the  strategy.  Tnese  lessons,  coupled  with  the  DoD 
Inspector  Generates  Audit,  "Effectiveness  of  the  Defense 
System  Acquisition  Review  Council  Process,"  and  the 
procedures  in  applicable  DoD  directives,  should  preclude  a 
similar  situation  from  occurring  in  the  future.  The  DoD 
Directive  5000.1,  for  example,  specifies  that  Milestone  II 
may  be  delayed  until  additional  development  has  been 
accomplished  to  provide  better  definition  of  the  system, 
thereby  reducing  further  risk  or  uncertainty.  Furthermore, 
DoD  Instruction  5000. s  and  the  DoD  Directive  5000.1 
establish  the  means  to  conduct  program  reviews  at  anytime  in 
the  acquisition  process .  Approval  of  an  accelerated 
acquisition  strategy  does  not  relieve  the  program's 
management  of  the  responsibility  and  the  requirement  to 
subject  the  decisions  specified  in  0MB  Circular  A-109, 

"Major  Systems  Acquisitions." 

RBCX)MMgMDAT10BI  3i  The  GAO  recommended  that  when  the  Army 
cons i clears  such  strategies  for  future  programs,  the  Secretary 
of  Defense  require  the  Army  to  ensure  the  strategy  gives 
decisionmakers  (1)  timely  evaluations  of  the  weapon's 
performance  based  on  adequate  and  realistic  operational 
testing  before  a  production  decision  is  made,  and 
(2)  sufficient  information  on  program  progress.  (p.  12,  GAO 
Draft  Report) 

DoD  POSITION;  Concur.  The  DOD  Directive  5000.1  specifies 
til  at  throughout  the  acquisition  process,  emphasis  shall  be 
placed  upon  verifying  actual  performance  through  test  and 
evaluation.  The  procedures  of  the  DoD  Directive  5000.3, 
"Test  and  Evaluation,"  will  be  integral  to  all  systems 
acquisition  planning  and  decision  making.  Additional 
emphasis  is  being  placed  upon  realistic,  timely  operational 
testing  and  adherence  to  documentation  required  by 
applicable  acquisition  directives.  The  revised  DoD 
Directive  5000.3,  incorporating  this  emphasis  in  accordance 
with  current  legislation,  is  projected  to  be  published 
by  March  1986.  Public  Law  98^94  become  effective  on 
September  24  1983  (subsequent  to  the  decision  for  SERGEANT 
YORK  to  enter  full  production),  and  stipulates  that  a  full- 
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production  decision  not  be  made  until  the  Director, 

Operational  Test  and  Evaluat ion,  has  submitted  a  report  to 
the  Secretary  of  Defense  and  the  Congress  stating  that  an 
adequate  test  of  the  system  was  conducted  and  the  results 
confirm  it  is  effective  and  combat  suitable.  Such  tests  and 
reports  became  available  on  the  SERGEANT  YORK  after  the 
system  was  in  production  and  formed  part  of  the 
determination  to  cancel  the  program.  The  Federal 
Acquisition  Regulations  give  guidance  in  selecting  the  type 
of  contract  as  a  function  of  risk.  The  choice  of  contract 
generally  proceeds  from  cost  reimbursement  to  fixed-price, 
as  development  translates  to  production.  The  DoD  Directive 
5000.1  and  DoD  Instruction  5000.2  establish  the  procedures 
to  obtain  sufficient  information  on  program  progress  for 
decisionmakers  both  through  scheduled  reviews  and  continuing 
staff  interaction  to  "Maintain  current  visibility  over 
matters  such  as  cost,  supportability ,  T&E,  industrial  base 
responsiveness,  and  production  readiness  throughout  the 
acquisition  process." 
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